

<!DOCTYPE html>
<html class="writer-html5" lang="en" >
<head>
  <meta charset="utf-8" />
  <meta name="generator" content="Docutils 0.19: https://docutils.sourceforge.io/" />

  <meta name="viewport" content="width=device-width, initial-scale=1.0" />
  
  <title>去重 &mdash; Ceph Documentation</title>
  

  
  <link rel="stylesheet" href="../../_static/ceph.css" type="text/css" />
  <link rel="stylesheet" href="../../_static/pygments.css" type="text/css" />
  <link rel="stylesheet" href="../../_static/pygments.css" type="text/css" />
  <link rel="stylesheet" href="../../_static/ceph.css" type="text/css" />
  <link rel="stylesheet" href="../../_static/graphviz.css" type="text/css" />
  <link rel="stylesheet" href="../../_static/css/custom.css" type="text/css" />

  
  

  
  

  

  
  <!--[if lt IE 9]>
    <script src="../../_static/js/html5shiv.min.js"></script>
  <![endif]-->
  
    
      <script type="text/javascript" id="documentation_options" data-url_root="../../" src="../../_static/documentation_options.js"></script>
        <script src="../../_static/jquery.js"></script>
        <script src="../../_static/_sphinx_javascript_frameworks_compat.js"></script>
        <script data-url_root="../../" id="documentation_options" src="../../_static/documentation_options.js"></script>
        <script src="../../_static/doctools.js"></script>
        <script src="../../_static/sphinx_highlight.js"></script>
    
    <script type="text/javascript" src="../../_static/js/theme.js"></script>

    
    <link rel="index" title="Index" href="../../genindex/" />
    <link rel="search" title="Search" href="../../search/" />
    <link rel="next" title="CephFS delayed deletion" href="../delayed-delete/" />
    <link rel="prev" title="C++17 and libstdc++ ABI" href="../cxx/" /> 
</head>

<body class="wy-body-for-nav">

   
  <header class="top-bar">
    <div role="navigation" aria-label="Page navigation">
  <ul class="wy-breadcrumbs">
      <li><a href="../../" class="icon icon-home" aria-label="Home"></a></li>
          <li class="breadcrumb-item"><a href="../internals/">Ceph 内幕</a></li>
      <li class="breadcrumb-item active">去重</li>
      <li class="wy-breadcrumbs-aside">
            <a href="../../_sources/dev/deduplication.rst.txt" rel="nofollow"> View page source</a>
      </li>
  </ul>
  <hr/>
</div>
  </header>
  <div class="wy-grid-for-nav">
    
    <nav data-toggle="wy-nav-shift" class="wy-nav-side">
      <div class="wy-side-scroll">
        <div class="wy-side-nav-search"  style="background: #eee" >
          

          
            <a href="../../" class="icon icon-home"> Ceph
          

          
          </a>

          

          
<div role="search">
  <form id="rtd-search-form" class="wy-form" action="../../search/" method="get">
    <input type="text" name="q" placeholder="Search docs" aria-label="Search docs" />
    <input type="hidden" name="check_keywords" value="yes" />
    <input type="hidden" name="area" value="default" />
  </form>
</div>

          
        </div>

        
        <div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation">
          
            
            
              
            
            
              <ul class="current">
<li class="toctree-l1"><a class="reference internal" href="../../start/">Ceph 简介</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../install/">安装 Ceph</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../cephadm/">Cephadm</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../rados/">Ceph 存储集群</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../cephfs/">Ceph 文件系统</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../rbd/">Ceph 块设备</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../radosgw/">Ceph 对象网关</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../mgr/">Ceph 管理器守护进程</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../mgr/dashboard/">Ceph 仪表盘</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../monitoring/">监控概览</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../api/">API 文档</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../architecture/">体系结构</a></li>
<li class="toctree-l1"><a class="reference internal" href="../developer_guide/">开发者指南</a></li>
<li class="toctree-l1 current"><a class="reference internal" href="../internals/">Ceph 内幕</a><ul class="current">
<li class="toctree-l2"><a class="reference internal" href="../balancer-design/">Ceph 如何均衡（读写、容量）</a></li>
<li class="toctree-l2"><a class="reference internal" href="../blkin/">Tracing Ceph With LTTng</a></li>
<li class="toctree-l2"><a class="reference internal" href="../blkin/#tracing-ceph-with-blkin">Tracing Ceph With Blkin</a></li>
<li class="toctree-l2"><a class="reference internal" href="../bluestore/">BlueStore Internals</a></li>
<li class="toctree-l2"><a class="reference internal" href="../ceph_krb_auth/">如何配置好 Ceph Kerberos 认证的详细文档</a></li>
<li class="toctree-l2"><a class="reference internal" href="../cephfs-mirroring/">CephFS Mirroring</a></li>
<li class="toctree-l2"><a class="reference internal" href="../cephfs-reclaim/">CephFS Reclaim Interface</a></li>
<li class="toctree-l2"><a class="reference internal" href="../cephfs-snapshots/">CephFS 快照</a></li>
<li class="toctree-l2"><a class="reference internal" href="../cephx/">Cephx</a></li>
<li class="toctree-l2"><a class="reference internal" href="../cephx_protocol/">Cephx 认证协议详细阐述</a></li>
<li class="toctree-l2"><a class="reference internal" href="../config/">配置管理系统</a></li>
<li class="toctree-l2"><a class="reference internal" href="../config-key/">config-key layout</a></li>
<li class="toctree-l2"><a class="reference internal" href="../context/">CephContext</a></li>
<li class="toctree-l2"><a class="reference internal" href="../continuous-integration/">Continuous Integration Architecture</a></li>
<li class="toctree-l2"><a class="reference internal" href="../corpus/">资料库结构</a></li>
<li class="toctree-l2"><a class="reference internal" href="../cpu-profiler/">Oprofile 的安装</a></li>
<li class="toctree-l2"><a class="reference internal" href="../crush-msr/">CRUSH MSR (Multi-step Retry)</a></li>
<li class="toctree-l2"><a class="reference internal" href="../cxx/">C++17 and libstdc++ ABI</a></li>
<li class="toctree-l2 current"><a class="current reference internal" href="#">去重</a><ul>
<li class="toctree-l3"><a class="reference internal" href="#id2">介绍</a></li>
<li class="toctree-l3"><a class="reference internal" href="#key-idea">Key Idea</a></li>
<li class="toctree-l3"><a class="reference internal" href="#design">Design</a></li>
<li class="toctree-l3"><a class="reference internal" href="#usage-patterns">Usage Patterns</a><ul>
<li class="toctree-l4"><a class="reference internal" href="#radosgw">RadosGW</a></li>
<li class="toctree-l4"><a class="reference internal" href="#rbd-cephfs">RBD/Cephfs</a></li>
</ul>
</li>
<li class="toctree-l3"><a class="reference internal" href="#rados-machinery">RADOS Machinery</a></li>
<li class="toctree-l3"><a class="reference internal" href="#status-and-future-work">Status and Future Work</a></li>
<li class="toctree-l3"><a class="reference internal" href="#how-to-use-deduplication">How to use deduplication</a><ul>
<li class="toctree-l4"><a class="reference internal" href="#prerequisite">Prerequisite</a></li>
<li class="toctree-l4"><a class="reference internal" href="#deatiled-instructions">Deatiled Instructions</a></li>
</ul>
</li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="../delayed-delete/">CephFS delayed deletion</a></li>
<li class="toctree-l2"><a class="reference internal" href="../dev_cluster_deployment/">开发集群的部署</a></li>
<li class="toctree-l2"><a class="reference internal" href="../dev_cluster_deployment/#id5">在同一机器上部署多套开发集群</a></li>
<li class="toctree-l2"><a class="reference internal" href="../development-workflow/">开发流程</a></li>
<li class="toctree-l2"><a class="reference internal" href="../documenting/">为 Ceph 写作文档</a></li>
<li class="toctree-l2"><a class="reference internal" href="../dpdk/">Ceph messenger DPDKStack</a></li>
<li class="toctree-l2"><a class="reference internal" href="../encoding/">序列化（编码、解码）</a></li>
<li class="toctree-l2"><a class="reference internal" href="../erasure-coded-pool/">纠删码存储池</a></li>
<li class="toctree-l2"><a class="reference internal" href="../file-striping/">File striping</a></li>
<li class="toctree-l2"><a class="reference internal" href="../freebsd/">FreeBSD Implementation details</a></li>
<li class="toctree-l2"><a class="reference internal" href="../generatedocs/">Ceph 文档的构建</a></li>
<li class="toctree-l2"><a class="reference internal" href="../health-reports/">Health Reports</a></li>
<li class="toctree-l2"><a class="reference internal" href="../iana/">IANA 号</a></li>
<li class="toctree-l2"><a class="reference internal" href="../kclient/">Testing changes to the Linux Kernel CephFS driver</a></li>
<li class="toctree-l2"><a class="reference internal" href="../kclient/#step-one-build-the-kernel">Step One: build the kernel</a></li>
<li class="toctree-l2"><a class="reference internal" href="../kclient/#step-two-create-a-vm">Step Two: create a VM</a></li>
<li class="toctree-l2"><a class="reference internal" href="../kclient/#step-three-networking-the-vm">Step Three: Networking the VM</a></li>
<li class="toctree-l2"><a class="reference internal" href="../kubernetes/">Hacking on Ceph in Kubernetes with Rook</a></li>
<li class="toctree-l2"><a class="reference internal" href="../libcephfs_proxy/">Design of the libcephfs proxy</a></li>
<li class="toctree-l2"><a class="reference internal" href="../libs/">库体系结构</a></li>
<li class="toctree-l2"><a class="reference internal" href="../logging/">集群日志的用法</a></li>
<li class="toctree-l2"><a class="reference internal" href="../logs/">调试日志</a></li>
<li class="toctree-l2"><a class="reference internal" href="../macos/">在 MacOS 上构建</a></li>
<li class="toctree-l2"><a class="reference internal" href="../mempool_accounting/">What is a mempool?</a></li>
<li class="toctree-l2"><a class="reference internal" href="../mempool_accounting/#some-common-mempools-that-we-can-track">Some common mempools that we can track</a></li>
<li class="toctree-l2"><a class="reference internal" href="../messenger/">Messenger notes</a></li>
<li class="toctree-l2"><a class="reference internal" href="../mon-bootstrap/">Monitor bootstrap</a></li>
<li class="toctree-l2"><a class="reference internal" href="../mon-elections/">Monitor Elections</a></li>
<li class="toctree-l2"><a class="reference internal" href="../mon-on-disk-formats/">ON-DISK FORMAT</a></li>
<li class="toctree-l2"><a class="reference internal" href="../mon-osdmap-prune/">FULL OSDMAP VERSION PRUNING</a></li>
<li class="toctree-l2"><a class="reference internal" href="../msgr2/">msgr2 协议（ msgr2.0 和 msgr2.1 ）</a></li>
<li class="toctree-l2"><a class="reference internal" href="../network-encoding/">Network Encoding</a></li>
<li class="toctree-l2"><a class="reference internal" href="../network-protocol/">网络协议</a></li>
<li class="toctree-l2"><a class="reference internal" href="../object-store/">对象存储架构概述</a></li>
<li class="toctree-l2"><a class="reference internal" href="../osd-class-path/">OSD class path issues</a></li>
<li class="toctree-l2"><a class="reference internal" href="../peering/">互联</a></li>
<li class="toctree-l2"><a class="reference internal" href="../perf/">Using perf</a></li>
<li class="toctree-l2"><a class="reference internal" href="../perf_counters/">性能计数器</a></li>
<li class="toctree-l2"><a class="reference internal" href="../perf_histograms/">Perf histograms</a></li>
<li class="toctree-l2"><a class="reference internal" href="../placement-group/">PG （归置组）说明</a></li>
<li class="toctree-l2"><a class="reference internal" href="../quick_guide/">开发者指南（快速）</a></li>
<li class="toctree-l2"><a class="reference internal" href="../rados-client-protocol/">RADOS 客户端协议</a></li>
<li class="toctree-l2"><a class="reference internal" href="../rbd-diff/">RBD 增量备份</a></li>
<li class="toctree-l2"><a class="reference internal" href="../rbd-export/">RBD Export &amp; Import</a></li>
<li class="toctree-l2"><a class="reference internal" href="../rbd-layering/">RBD Layering</a></li>
<li class="toctree-l2"><a class="reference internal" href="../release-checklists/">Release checklists</a></li>
<li class="toctree-l2"><a class="reference internal" href="../release-process/">Ceph Release Process</a></li>
<li class="toctree-l2"><a class="reference internal" href="../seastore/">SeaStore</a></li>
<li class="toctree-l2"><a class="reference internal" href="../sepia/">Sepia 社区测试实验室</a></li>
<li class="toctree-l2"><a class="reference internal" href="../session_authentication/">Session Authentication for the Cephx Protocol</a></li>
<li class="toctree-l2"><a class="reference internal" href="../testing/">测试笔记</a></li>
<li class="toctree-l2"><a class="reference internal" href="../versions/">Public OSD Version</a></li>
<li class="toctree-l2"><a class="reference internal" href="../vstart-ganesha/">NFS CephFS-RGW Developer Guide</a></li>
<li class="toctree-l2"><a class="reference internal" href="../wireshark/">Wireshark Dissector</a></li>
<li class="toctree-l2"><a class="reference internal" href="../zoned-storage/">Zoned Storage Support</a></li>
<li class="toctree-l2"><a class="reference internal" href="../osd_internals/">OSD 开发者文档</a></li>
<li class="toctree-l2"><a class="reference internal" href="../mds_internals/">MDS 开发者文档</a></li>
<li class="toctree-l2"><a class="reference internal" href="../radosgw/">RADOS 网关开发者文档</a></li>
<li class="toctree-l2"><a class="reference internal" href="../ceph-volume/">ceph-volume 开发者文档</a></li>
<li class="toctree-l2"><a class="reference internal" href="../crimson/">Crimson developer documentation</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../../governance/">项目管理</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../foundation/">Ceph 基金会</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../ceph-volume/">ceph-volume</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../releases/general/">Ceph 版本（总目录）</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../releases/">Ceph 版本（索引）</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../security/">Security</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../hardware-monitoring/">硬件监控</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../glossary/">Ceph 术语</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../jaegertracing/">Tracing</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../translation_cn/">中文版翻译资源</a></li>
</ul>

            
          
        </div>
        
      </div>
    </nav>

    <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap">

      
      <nav class="wy-nav-top" aria-label="top navigation">
        
          <i data-toggle="wy-nav-top" class="fa fa-bars"></i>
          <a href="../../">Ceph</a>
        
      </nav>


      <div class="wy-nav-content">
        
        <div class="rst-content">
        
          <div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
           <div itemprop="articleBody">
            
<div id="dev-warning" class="admonition note">
  <p class="first admonition-title">Notice</p>
  <p class="last">This document is for a development version of Ceph.</p>
</div>
  <div id="docubetter" align="right" style="padding: 5px; font-weight: bold;">
    <a href="https://pad.ceph.com/p/Report_Documentation_Bugs">Report a Documentation Bug</a>
  </div>

  
  <section id="id1">
<h1>去重<a class="headerlink" href="#id1" title="Permalink to this heading"></a></h1>
<section id="id2">
<h2>介绍<a class="headerlink" href="#id2" title="Permalink to this heading"></a></h2>
<p>Applying data deduplication on an existing software stack is not easy
due to additional metadata management and original data processing
procedure.</p>
<p>In a typical deduplication system, the input source as a data
object is split into multiple chunks by a chunking algorithm.
The deduplication system then compares each chunk with
the existing data chunks, stored in the storage previously.
To this end, a fingerprint index that stores the hash value
of each chunk is employed by the deduplication system
in order to easily find the existing chunks by comparing
hash value rather than searching all contents that reside in
the underlying storage.</p>
<p>There are many challenges in order to implement deduplication on top
of Ceph. Among them, two issues are essential for deduplication.
First is managing scalability of fingerprint index; Second is
it is complex to ensure compatibility between newly applied
deduplication metadata and existing metadata.</p>
</section>
<section id="key-idea">
<h2>Key Idea<a class="headerlink" href="#key-idea" title="Permalink to this heading"></a></h2>
<p>1. Content hashing (Double hashing): Each client can find an object data
for an object ID using CRUSH. With CRUSH, a client knows object’s location
in Base tier.
By hashing object’s content at Base tier, a new OID (chunk ID) is generated.
Chunk tier stores in the new OID that has a partial content of original object.</p>
<blockquote>
<div><p>Client 1 -&gt; OID=1 -&gt; HASH(1’s content)=K -&gt; OID=K -&gt;
CRUSH(K) -&gt; chunk’s location</p>
</div></blockquote>
<p>2. Self-contained object: The external metadata design
makes difficult for integration with storage feature support
since existing storage features cannot recognize the
additional external data structures. If we can design data
deduplication system without any external component, the
original storage features can be reused.</p>
<p>More details in <a class="reference external" href="https://ieeexplore.ieee.org/document/8416369">https://ieeexplore.ieee.org/document/8416369</a></p>
</section>
<section id="design">
<h2>Design<a class="headerlink" href="#design" title="Permalink to this heading"></a></h2>
<p class="ditaa">
<img src="../../_images/ditaa-91d356ef36a6040ed214965c6937bd2722261086.png"/>
</p>
<p>Pool-based object management:
We define two pools.
The metadata pool stores metadata objects and the chunk pool stores
chunk objects. Since these two pools are divided based on
the purpose and usage, each pool can be managed more
efficiently according to its different characteristics. Base
pool and the chunk pool can separately select a redundancy
scheme between replication and erasure coding depending on
its usage and each pool can be placed in a different storage
location depending on the required performance.</p>
<p>Regarding how to use, please see <code class="docutils literal notranslate"><span class="pre">osd_internals/manifest.rst</span></code></p>
</section>
<section id="usage-patterns">
<h2>Usage Patterns<a class="headerlink" href="#usage-patterns" title="Permalink to this heading"></a></h2>
<p>The different Ceph interface layers present potentially different oportunities
and costs for deduplication and tiering in general.</p>
<section id="radosgw">
<h3>RadosGW<a class="headerlink" href="#radosgw" title="Permalink to this heading"></a></h3>
<p>S3 big data workloads seem like a good opportunity for deduplication.  These
objects tend to be write once, read mostly objects which don’t see partial
overwrites.  As such, it makes sense to fingerprint and dedup up front.</p>
<p>Unlike cephfs and rbd, radosgw has a system for storing
explicit metadata in the head object of a logical s3 object for
locating the remaining pieces.  As such, radosgw could use the
refcounting machinery (<code class="docutils literal notranslate"><span class="pre">osd_internals/refcount.rst</span></code>) directly without
needing direct support from rados for manifests.</p>
</section>
<section id="rbd-cephfs">
<h3>RBD/Cephfs<a class="headerlink" href="#rbd-cephfs" title="Permalink to this heading"></a></h3>
<p>RBD and CephFS both use deterministic naming schemes to partition
block devices/file data over rados objects.  As such, the redirection
metadata would need to be included as part of rados, presumably
transparently.</p>
<p>Moreover, unlike radosgw, rbd/cephfs rados objects can see overwrites.
For those objects, we don’t really want to perform dedup, and we don’t
want to pay a write latency penalty in the hot path to do so anyway.
As such, performing tiering and dedup on cold objects in the background
is likely to be preferred.</p>
<p>One important wrinkle, however, is that both rbd and cephfs workloads
often feature usage of snapshots.  This means that the rados manifest
support needs robust support for snapshots.</p>
</section>
</section>
<section id="rados-machinery">
<h2>RADOS Machinery<a class="headerlink" href="#rados-machinery" title="Permalink to this heading"></a></h2>
<p>For more information on rados redirect/chunk/dedup support, see <code class="docutils literal notranslate"><span class="pre">osd_internals/manifest.rst</span></code>.
For more information on rados refcount support, see <code class="docutils literal notranslate"><span class="pre">osd_internals/refcount.rst</span></code>.</p>
</section>
<section id="status-and-future-work">
<h2>Status and Future Work<a class="headerlink" href="#status-and-future-work" title="Permalink to this heading"></a></h2>
<p>At the moment, there exists some preliminary support for manifest
objects within the OSD as well as a dedup tool.</p>
<p>RadosGW data warehouse workloads probably represent the largest
opportunity for this feature, so the first priority is probably to add
direct support for fingerprinting and redirects into the refcount pool
to radosgw.</p>
<p>Aside from radosgw, completing work on manifest object support in the
OSD particularly as it relates to snapshots would be the next step for
rbd and cephfs workloads.</p>
</section>
<section id="how-to-use-deduplication">
<h2>How to use deduplication<a class="headerlink" href="#how-to-use-deduplication" title="Permalink to this heading"></a></h2>
<blockquote>
<div><ul class="simple">
<li><p>This feature is highly experimental and is subject to change or removal.</p></li>
</ul>
</div></blockquote>
<p>Ceph provides deduplication using RADOS machinery.
Below we explain how to perform deduplication.</p>
<section id="prerequisite">
<h3>Prerequisite<a class="headerlink" href="#prerequisite" title="Permalink to this heading"></a></h3>
<p>If the Ceph cluster is started from Ceph mainline, users need to check
<code class="docutils literal notranslate"><span class="pre">ceph-test</span></code> package which is including ceph-dedup-tool is installed.</p>
</section>
<section id="deatiled-instructions">
<h3>Deatiled Instructions<a class="headerlink" href="#deatiled-instructions" title="Permalink to this heading"></a></h3>
<p>Users can use ceph-dedup-tool with <code class="docutils literal notranslate"><span class="pre">estimate</span></code>, <code class="docutils literal notranslate"><span class="pre">sample-dedup</span></code>,
<code class="docutils literal notranslate"><span class="pre">chunk-scrub</span></code>, and <code class="docutils literal notranslate"><span class="pre">chunk-repair</span></code> operations. To provide better
convenience for users, we have enabled necessary operations through
ceph-dedup-tool, and we recommend using the following operations freely
by using any types of scripts.</p>
<section id="estimate-space-saving-ratio-of-a-target-pool-using-ceph-dedup-tool">
<h4>1. Estimate space saving ratio of a target pool using <code class="docutils literal notranslate"><span class="pre">ceph-dedup-tool</span></code>.<a class="headerlink" href="#estimate-space-saving-ratio-of-a-target-pool-using-ceph-dedup-tool" title="Permalink to this heading"></a></h4>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>ceph-dedup-tool<span class="w"> </span>--op<span class="w"> </span>estimate
<span class="w">  </span>--pool<span class="w"> </span><span class="o">[</span>BASE_POOL<span class="o">]</span>
<span class="w">  </span>--chunk-size<span class="w"> </span><span class="o">[</span>CHUNK_SIZE<span class="o">]</span>
<span class="w">  </span>--chunk-algorithm<span class="w"> </span><span class="o">[</span>fixed<span class="p">|</span>fastcdc<span class="o">]</span>
<span class="w">  </span>--fingerprint-algorithm<span class="w"> </span><span class="o">[</span>sha1<span class="p">|</span>sha256<span class="p">|</span>sha512<span class="o">]</span>
<span class="w">  </span>--max-thread<span class="w"> </span><span class="o">[</span>THREAD_COUNT<span class="o">]</span>
</pre></div>
</div>
<p>This CLI command will show how much storage space can be saved when deduplication
is applied on the pool. If the amount of the saved space is higher than user’s expectation,
the pool probably is worth performing deduplication.
Users should specify the <code class="docutils literal notranslate"><span class="pre">BASE_POOL</span></code>, within which the object targeted for deduplication
is stored. The users also need to run ceph-dedup-tool multiple time
with varying <code class="docutils literal notranslate"><span class="pre">chunk_size</span></code> to find the optimal chunk size. Note that the
optimal value probably differs in the content of each object in case of fastcdc
chunk algorithm (not fixed).</p>
<p>Example output:</p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span><span class="o">{</span>
<span class="w">  </span><span class="s2">&quot;chunk_algo&quot;</span>:<span class="w"> </span><span class="s2">&quot;fastcdc&quot;</span>,
<span class="w">  </span><span class="s2">&quot;chunk_sizes&quot;</span>:<span class="w"> </span><span class="o">[</span>
<span class="w">    </span><span class="o">{</span>
<span class="w">      </span><span class="s2">&quot;target_chunk_size&quot;</span>:<span class="w"> </span><span class="m">8192</span>,
<span class="w">      </span><span class="s2">&quot;dedup_bytes_ratio&quot;</span>:<span class="w"> </span><span class="m">0</span>.4897049
<span class="w">      </span><span class="s2">&quot;dedup_object_ratio&quot;</span>:<span class="w"> </span><span class="m">34</span>.567315
<span class="w">      </span><span class="s2">&quot;chunk_size_average&quot;</span>:<span class="w"> </span><span class="m">64439</span>,
<span class="w">      </span><span class="s2">&quot;chunk_size_stddev&quot;</span>:<span class="w"> </span><span class="m">33620</span>
<span class="w">    </span><span class="o">}</span>
<span class="w">  </span><span class="o">]</span>,
<span class="w">  </span><span class="s2">&quot;summary&quot;</span>:<span class="w"> </span><span class="o">{</span>
<span class="w">    </span><span class="s2">&quot;examined_objects&quot;</span>:<span class="w"> </span><span class="m">95</span>,
<span class="w">    </span><span class="s2">&quot;examined_bytes&quot;</span>:<span class="w"> </span><span class="m">214968649</span>
<span class="w">  </span><span class="o">}</span>
<span class="o">}</span>
</pre></div>
</div>
<p>The above is an example output when executing <code class="docutils literal notranslate"><span class="pre">estimate</span></code>. <code class="docutils literal notranslate"><span class="pre">target_chunk_size</span></code> is the same as
<code class="docutils literal notranslate"><span class="pre">chunk_size</span></code> given by the user. <code class="docutils literal notranslate"><span class="pre">dedup_bytes_ratio</span></code> shows how many bytes are redundant from
examined bytes. For instance, 1 - <code class="docutils literal notranslate"><span class="pre">dedup_bytes_ratio</span></code> means the percentage of saved storage space.
<code class="docutils literal notranslate"><span class="pre">dedup_object_ratio</span></code> is the generated chunk objects / <code class="docutils literal notranslate"><span class="pre">examined_objects</span></code>. <code class="docutils literal notranslate"><span class="pre">chunk_size_average</span></code>
means that the divided chunk size on average when performing CDC---this may differnet from <code class="docutils literal notranslate"><span class="pre">target_chunk_size</span></code>
because CDC genarates different chunk-boundary depending on the content. <code class="docutils literal notranslate"><span class="pre">chunk_size_stddev</span></code>
represents the standard deviation of the chunk size.</p>
</section>
<section id="create-chunk-pool">
<h4>2. Create chunk pool.<a class="headerlink" href="#create-chunk-pool" title="Permalink to this heading"></a></h4>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>ceph<span class="w"> </span>osd<span class="w"> </span>pool<span class="w"> </span>create<span class="w"> </span><span class="o">[</span>CHUNK_POOL<span class="o">]</span>
</pre></div>
</div>
</section>
<section id="run-dedup-command-there-are-two-ways">
<h4>3. Run dedup command (there are two ways).<a class="headerlink" href="#run-dedup-command-there-are-two-ways" title="Permalink to this heading"></a></h4>
<ul class="simple">
<li><p><strong>sample-dedup</strong></p></li>
</ul>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>ceph-dedup-tool<span class="w"> </span>--op<span class="w"> </span>sample-dedup
<span class="w">  </span>--pool<span class="w"> </span><span class="o">[</span>BASE_POOL<span class="o">]</span>
<span class="w">  </span>--chunk-pool<span class="w"> </span><span class="o">[</span>CHUNK_POOL<span class="o">]</span>
<span class="w">  </span>--chunk-size<span class="w"> </span><span class="o">[</span>CHUNK_SIZE<span class="o">]</span>
<span class="w">  </span>--chunk-algorithm<span class="w"> </span><span class="o">[</span>fastcdc<span class="o">]</span>
<span class="w">  </span>--fingerprint-algorithm<span class="w"> </span><span class="o">[</span>sha1<span class="p">|</span>sha256<span class="p">|</span>sha512<span class="o">]</span>
<span class="w">  </span>--chunk-dedup-threshold<span class="w"> </span><span class="o">[</span>THRESHOLD<span class="o">]</span>
<span class="w">  </span>--max-thread<span class="w"> </span><span class="o">[</span>THREAD_COUNT<span class="o">]</span>
<span class="w">  </span>--sampling-ratio<span class="w"> </span><span class="o">[</span>SAMPLE_RATIO<span class="o">]</span>
<span class="w">  </span>--wakeup-period<span class="w"> </span><span class="o">[</span>WAKEUP_PERIOD<span class="o">]</span>
<span class="w">  </span>--loop
<span class="w">  </span>--snap
</pre></div>
</div>
<p>The <code class="docutils literal notranslate"><span class="pre">sample-dedup</span></code> comamnd spawns threads specified by <code class="docutils literal notranslate"><span class="pre">THREAD_COUNT</span></code> to deduplicate objects on
the <code class="docutils literal notranslate"><span class="pre">BASE_POOL</span></code>. According to sampling-ratio---do a full search if <code class="docutils literal notranslate"><span class="pre">SAMPLE_RATIO</span></code> is 100, the threads selectively
perform deduplication if the chunk is redundant over <code class="docutils literal notranslate"><span class="pre">THRESHOLD</span></code> times during iteration.
If --loop is set, the theads will wakeup after <code class="docutils literal notranslate"><span class="pre">WAKEUP_PERIOD</span></code>. If not, the threads will exit after one iteration.</p>
<p>Example output:</p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>$<span class="w"> </span>bin/ceph<span class="w"> </span>df
---<span class="w"> </span>RAW<span class="w"> </span>STORAGE<span class="w"> </span>---
CLASS<span class="w">     </span>SIZE<span class="w">    </span>AVAIL<span class="w">     </span>USED<span class="w">  </span>RAW<span class="w"> </span>USED<span class="w">  </span>%RAW<span class="w"> </span>USED
ssd<span class="w">    </span><span class="m">303</span><span class="w"> </span>GiB<span class="w">  </span><span class="m">294</span><span class="w"> </span>GiB<span class="w">  </span><span class="m">9</span>.0<span class="w"> </span>GiB<span class="w">   </span><span class="m">9</span>.0<span class="w"> </span>GiB<span class="w">       </span><span class="m">2</span>.99
TOTAL<span class="w">  </span><span class="m">303</span><span class="w"> </span>GiB<span class="w">  </span><span class="m">294</span><span class="w"> </span>GiB<span class="w">  </span><span class="m">9</span>.0<span class="w"> </span>GiB<span class="w">   </span><span class="m">9</span>.0<span class="w"> </span>GiB<span class="w">       </span><span class="m">2</span>.99

---<span class="w"> </span>POOLS<span class="w"> </span>---
POOL<span class="w">   </span>ID<span class="w">  </span>PGS<span class="w">   </span>STORED<span class="w">  </span>OBJECTS<span class="w">     </span>USED<span class="w">  </span>%USED<span class="w">  </span>MAX<span class="w"> </span>AVAIL
.mgr<span class="w">    </span><span class="m">1</span><span class="w">    </span><span class="m">1</span><span class="w">  </span><span class="m">577</span><span class="w"> </span>KiB<span class="w">        </span><span class="m">2</span><span class="w">  </span><span class="m">1</span>.7<span class="w"> </span>MiB<span class="w">      </span><span class="m">0</span><span class="w">     </span><span class="m">97</span><span class="w"> </span>GiB
base<span class="w">    </span><span class="m">2</span><span class="w">   </span><span class="m">32</span><span class="w">  </span><span class="m">2</span>.0<span class="w"> </span>GiB<span class="w">      </span><span class="m">517</span><span class="w">  </span><span class="m">6</span>.0<span class="w"> </span>GiB<span class="w">   </span><span class="m">2</span>.02<span class="w">     </span><span class="m">97</span><span class="w"> </span>GiB
chunk<span class="w">   </span><span class="m">3</span><span class="w">   </span><span class="m">32</span><span class="w">   </span><span class="m">0</span><span class="w">  </span>B<span class="w">          </span><span class="m">0</span><span class="w">    </span><span class="m">0</span><span class="w">   </span>B<span class="w">      </span><span class="m">0</span><span class="w">     </span><span class="m">97</span><span class="w"> </span>GiB

$<span class="w"> </span>bin/ceph-dedup-tool<span class="w"> </span>--op<span class="w"> </span>sample-dedup<span class="w"> </span>--pool<span class="w"> </span>base<span class="w"> </span>--chunk-pool<span class="w"> </span>chunk
<span class="w">  </span>--fingerprint-algorithm<span class="w"> </span>sha1<span class="w"> </span>--chunk-algorithm<span class="w"> </span>fastcdc<span class="w"> </span>--loop<span class="w"> </span>--sampling-ratio<span class="w"> </span><span class="m">100</span>
<span class="w">  </span>--chunk-dedup-threshold<span class="w"> </span><span class="m">2</span><span class="w"> </span>--chunk-size<span class="w"> </span><span class="m">8192</span><span class="w"> </span>--max-thread<span class="w"> </span><span class="m">4</span><span class="w"> </span>--wakeup-period<span class="w"> </span><span class="m">60</span>

$<span class="w"> </span>bin/ceph<span class="w"> </span>df
---<span class="w"> </span>RAW<span class="w"> </span>STORAGE<span class="w"> </span>---
CLASS<span class="w">     </span>SIZE<span class="w">    </span>AVAIL<span class="w">     </span>USED<span class="w">  </span>RAW<span class="w"> </span>USED<span class="w">  </span>%RAW<span class="w"> </span>USED
ssd<span class="w">    </span><span class="m">303</span><span class="w"> </span>GiB<span class="w">  </span><span class="m">298</span><span class="w"> </span>GiB<span class="w">  </span><span class="m">5</span>.4<span class="w"> </span>GiB<span class="w">   </span><span class="m">5</span>.4<span class="w"> </span>GiB<span class="w">       </span><span class="m">1</span>.80
TOTAL<span class="w">  </span><span class="m">303</span><span class="w"> </span>GiB<span class="w">  </span><span class="m">298</span><span class="w"> </span>GiB<span class="w">  </span><span class="m">5</span>.4<span class="w"> </span>GiB<span class="w">   </span><span class="m">5</span>.4<span class="w"> </span>GiB<span class="w">       </span><span class="m">1</span>.80

---<span class="w"> </span>POOLS<span class="w"> </span>---
POOL<span class="w">   </span>ID<span class="w">  </span>PGS<span class="w">   </span>STORED<span class="w">  </span>OBJECTS<span class="w">     </span>USED<span class="w">  </span>%USED<span class="w">  </span>MAX<span class="w"> </span>AVAIL
.mgr<span class="w">    </span><span class="m">1</span><span class="w">    </span><span class="m">1</span><span class="w">  </span><span class="m">577</span><span class="w"> </span>KiB<span class="w">        </span><span class="m">2</span><span class="w">  </span><span class="m">1</span>.7<span class="w"> </span>MiB<span class="w">      </span><span class="m">0</span><span class="w">     </span><span class="m">98</span><span class="w"> </span>GiB
base<span class="w">    </span><span class="m">2</span><span class="w">   </span><span class="m">32</span><span class="w">  </span><span class="m">452</span><span class="w"> </span>MiB<span class="w">      </span><span class="m">262</span><span class="w">  </span><span class="m">1</span>.3<span class="w"> </span>GiB<span class="w">   </span><span class="m">0</span>.50<span class="w">     </span><span class="m">98</span><span class="w"> </span>GiB
chunk<span class="w">   </span><span class="m">3</span><span class="w">   </span><span class="m">32</span><span class="w">  </span><span class="m">258</span><span class="w"> </span>MiB<span class="w">   </span><span class="m">25</span>.91k<span class="w">  </span><span class="m">938</span><span class="w"> </span>MiB<span class="w">   </span><span class="m">0</span>.31<span class="w">     </span><span class="m">98</span><span class="w"> </span>GiB
</pre></div>
</div>
<ul class="simple">
<li><p><strong>object dedup</strong></p></li>
</ul>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>ceph-dedup-tool<span class="w"> </span>--op<span class="w"> </span>object-dedup
<span class="w">  </span>--pool<span class="w"> </span><span class="o">[</span>BASE_POOL<span class="o">]</span>
<span class="w">  </span>--object<span class="w"> </span><span class="o">[</span>OID<span class="o">]</span>
<span class="w">  </span>--chunk-pool<span class="w"> </span><span class="o">[</span>CHUNK_POOL<span class="o">]</span>
<span class="w">  </span>--fingerprint-algorithm<span class="w"> </span><span class="o">[</span>sha1<span class="p">|</span>sha256<span class="p">|</span>sha512<span class="o">]</span>
<span class="w">  </span>--dedup-cdc-chunk-size<span class="w"> </span><span class="o">[</span>CHUNK_SIZE<span class="o">]</span>
</pre></div>
</div>
<p>The <code class="docutils literal notranslate"><span class="pre">object-dedup</span></code> command triggers deduplication on the RADOS object specified by <code class="docutils literal notranslate"><span class="pre">OID</span></code>.
All parameters shown above must be specified. <code class="docutils literal notranslate"><span class="pre">CHUNK_SIZE</span></code> should be taken from
the results of step 1 above.
Note that when this command is executed, <code class="docutils literal notranslate"><span class="pre">fastcdc</span></code> will be set by default and other parameters
such as <code class="docutils literal notranslate"><span class="pre">fingerprint-algorithm</span></code> and <code class="docutils literal notranslate"><span class="pre">CHUNK_SIZE</span></code> will be set as defaults for the pool.
Deduplicated objects will appear in the chunk pool. If the object is mutated over time, user needs to re-run
<code class="docutils literal notranslate"><span class="pre">object-dedup</span></code> because chunk-boundary should be recalculated based on updated contents.
The user needs to specify <code class="docutils literal notranslate"><span class="pre">snap</span></code> if the target object is snapshotted. After deduplication is done, the target
object size in <code class="docutils literal notranslate"><span class="pre">BASE_POOL</span></code> is zero (evicted) and chunks objects are genereated---these appear in <code class="docutils literal notranslate"><span class="pre">CHUNK_POOL</span></code>.</p>
</section>
<section id="read-write-i-os">
<h4>4. Read/write I/Os<a class="headerlink" href="#read-write-i-os" title="Permalink to this heading"></a></h4>
<p>After step 3, the users don’t need to consider anything about I/Os. Deduplicated objects are
completely compatible with existing RADOS operations.</p>
</section>
<section id="run-scrub-to-fix-reference-count">
<h4>5. Run scrub to fix reference count<a class="headerlink" href="#run-scrub-to-fix-reference-count" title="Permalink to this heading"></a></h4>
<p>Reference mismatches can on rare occasions occur to false positives when handling reference counts for
deduplicated RADOS objects. These mismatches will be fixed by periodically scrubbing the pool:</p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>ceph-dedup-tool<span class="w"> </span>--op<span class="w"> </span>chunk-scrub
<span class="w">  </span>--chunk-pool<span class="w"> </span><span class="o">[</span>CHUNK_POOL<span class="o">]</span>
<span class="w">  </span>--pool<span class="w"> </span><span class="o">[</span>POOL<span class="o">]</span>
<span class="w">  </span>--max-thread<span class="w"> </span><span class="o">[</span>THREAD_COUNT<span class="o">]</span>
</pre></div>
</div>
<p>The <code class="docutils literal notranslate"><span class="pre">chunk-scrub</span></code> command identifies reference mismatches between a
metadata object and a chunk object. The <code class="docutils literal notranslate"><span class="pre">chunk-pool</span></code> parameter tells
where the target chunk objects are located to the ceph-dedup-tool.</p>
<p>Example output:</p>
<p>A reference mismatch is intentionally created by inserting a reference (dummy-obj) into a chunk object (2ac67f70d3dd187f8f332bb1391f61d4e5c9baae) by using chunk-get-ref.</p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>$<span class="w"> </span>bin/ceph-dedup-tool<span class="w"> </span>--op<span class="w"> </span>dump-chunk-refs<span class="w"> </span>--chunk-pool<span class="w"> </span>chunk<span class="w"> </span>--object<span class="w"> </span>2ac67f70d3dd187f8f332bb1391f61d4e5c9baae
<span class="o">{</span>
<span class="w">  </span><span class="s2">&quot;type&quot;</span>:<span class="w"> </span><span class="s2">&quot;by_object&quot;</span>,
<span class="w">  </span><span class="s2">&quot;count&quot;</span>:<span class="w"> </span><span class="m">2</span>,
<span class="w">    </span><span class="s2">&quot;refs&quot;</span>:<span class="w"> </span><span class="o">[</span>
<span class="w">    </span><span class="o">{</span>
<span class="w">      </span><span class="s2">&quot;oid&quot;</span>:<span class="w"> </span><span class="s2">&quot;testfile2&quot;</span>,
<span class="w">            </span><span class="s2">&quot;key&quot;</span>:<span class="w"> </span><span class="s2">&quot;&quot;</span>,
<span class="w">            </span><span class="s2">&quot;snapid&quot;</span>:<span class="w"> </span>-2,
<span class="w">            </span><span class="s2">&quot;hash&quot;</span>:<span class="w"> </span><span class="m">2905889452</span>,
<span class="w">            </span><span class="s2">&quot;max&quot;</span>:<span class="w"> </span><span class="m">0</span>,
<span class="w">            </span><span class="s2">&quot;pool&quot;</span>:<span class="w"> </span><span class="m">2</span>,
<span class="w">            </span><span class="s2">&quot;namespace&quot;</span>:<span class="w"> </span><span class="s2">&quot;&quot;</span>
<span class="w">    </span><span class="o">}</span>,
<span class="w">    </span><span class="o">{</span>
<span class="w">      </span><span class="s2">&quot;oid&quot;</span>:<span class="w"> </span><span class="s2">&quot;dummy-obj&quot;</span>,
<span class="w">      </span><span class="s2">&quot;key&quot;</span>:<span class="w"> </span><span class="s2">&quot;&quot;</span>,
<span class="w">      </span><span class="s2">&quot;snapid&quot;</span>:<span class="w"> </span>-2,
<span class="w">      </span><span class="s2">&quot;hash&quot;</span>:<span class="w"> </span><span class="m">1203585162</span>,
<span class="w">      </span><span class="s2">&quot;max&quot;</span>:<span class="w"> </span><span class="m">0</span>,
<span class="w">      </span><span class="s2">&quot;pool&quot;</span>:<span class="w"> </span><span class="m">2</span>,
<span class="w">      </span><span class="s2">&quot;namespace&quot;</span>:<span class="w"> </span><span class="s2">&quot;&quot;</span>
<span class="w">    </span><span class="o">}</span>
<span class="w">  </span><span class="o">]</span>
<span class="o">}</span>

$<span class="w"> </span>bin/ceph-dedup-tool<span class="w"> </span>--op<span class="w"> </span>chunk-scrub<span class="w"> </span>--chunk-pool<span class="w"> </span>chunk<span class="w"> </span>--max-thread<span class="w"> </span><span class="m">10</span>
<span class="m">10</span><span class="w"> </span>seconds<span class="w"> </span>is<span class="w"> </span><span class="nb">set</span><span class="w"> </span>as<span class="w"> </span>report<span class="w"> </span>period<span class="w"> </span>by<span class="w"> </span>default
join
join
2ac67f70d3dd187f8f332bb1391f61d4e5c9baae
--done--
2ac67f70d3dd187f8f332bb1391f61d4e5c9baae<span class="w"> </span>ref<span class="w"> </span><span class="m">10</span>:5102bde2:::dummy-obj:head:<span class="w"> </span>referencing<span class="w"> </span>pool<span class="w"> </span>does<span class="w"> </span>not<span class="w"> </span>exist
--done--
<span class="w"> </span>Total<span class="w"> </span>object<span class="w"> </span>:<span class="w"> </span><span class="m">1</span>
<span class="w"> </span>Examined<span class="w"> </span>object<span class="w"> </span>:<span class="w"> </span><span class="m">1</span>
<span class="w"> </span>Damaged<span class="w"> </span>object<span class="w"> </span>:<span class="w"> </span><span class="m">1</span>
</pre></div>
</div>
</section>
<section id="repair-a-mismatched-chunk-reference">
<h4>6. Repair a mismatched chunk reference<a class="headerlink" href="#repair-a-mismatched-chunk-reference" title="Permalink to this heading"></a></h4>
<p>If any reference mismatches occur after the <code class="docutils literal notranslate"><span class="pre">chunk-scrub</span></code>, it is
recommended to perform the <code class="docutils literal notranslate"><span class="pre">chunk-repair</span></code> operation to fix reference
mismatches. The <code class="docutils literal notranslate"><span class="pre">chunk-repair</span></code> operation helps in resolving the
reference mismatch and restoring consistency.</p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>ceph-dedup-tool<span class="w"> </span>--op<span class="w"> </span>chunk-repair
<span class="w">  </span>--chunk-pool<span class="w"> </span><span class="o">[</span>CHUNK_POOL_NAME<span class="o">]</span>
<span class="w">  </span>--object<span class="w"> </span><span class="o">[</span>CHUNK_OID<span class="o">]</span>
<span class="w">  </span>--target-ref<span class="w"> </span><span class="o">[</span>TARGET_OID<span class="o">]</span>
<span class="w">  </span>--target-ref-pool-id<span class="w"> </span><span class="o">[</span>TARGET_POOL_ID<span class="o">]</span>
</pre></div>
</div>
<p><code class="docutils literal notranslate"><span class="pre">chunk-repair</span></code> fixes the <code class="docutils literal notranslate"><span class="pre">target-ref</span></code>, which is a wrong reference of
an <code class="docutils literal notranslate"><span class="pre">object</span></code>. To fix it correctly, the users must enter the correct
<code class="docutils literal notranslate"><span class="pre">TARGET_OID</span></code> and <code class="docutils literal notranslate"><span class="pre">TARGET_POOL_ID</span></code>.</p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>$<span class="w"> </span>bin/ceph-dedup-tool<span class="w"> </span>--op<span class="w"> </span>chunk-repair<span class="w"> </span>--chunk-pool<span class="w"> </span>chunk<span class="w"> </span>--object<span class="w"> </span>2ac67f70d3dd187f8f332bb1391f61d4e5c9baae<span class="w"> </span>--target-ref<span class="w"> </span>dummy-obj<span class="w"> </span>--target-ref-pool-id<span class="w"> </span><span class="m">10</span>
2ac67f70d3dd187f8f332bb1391f61d4e5c9baae<span class="w"> </span>has<span class="w"> </span><span class="m">1</span><span class="w"> </span>references<span class="w"> </span><span class="k">for</span><span class="w"> </span>dummy-obj
dummy-obj<span class="w"> </span>has<span class="w"> </span><span class="m">0</span><span class="w"> </span>references<span class="w"> </span><span class="k">for</span><span class="w"> </span>2ac67f70d3dd187f8f332bb1391f61d4e5c9baae
<span class="w"> </span>fix<span class="w"> </span>dangling<span class="w"> </span>reference<span class="w"> </span>from<span class="w"> </span><span class="m">1</span><span class="w"> </span>to<span class="w"> </span><span class="m">0</span>

$<span class="w"> </span>bin/ceph-dedup-tool<span class="w"> </span>--op<span class="w"> </span>dump-chunk-refs<span class="w"> </span>--chunk-pool<span class="w"> </span>chunk<span class="w"> </span>--object<span class="w"> </span>2ac67f70d3dd187f8f332bb1391f61d4e5c9baae
<span class="o">{</span>
<span class="w">  </span><span class="s2">&quot;type&quot;</span>:<span class="w"> </span><span class="s2">&quot;by_object&quot;</span>,
<span class="w">  </span><span class="s2">&quot;count&quot;</span>:<span class="w"> </span><span class="m">1</span>,
<span class="w">  </span><span class="s2">&quot;refs&quot;</span>:<span class="w"> </span><span class="o">[</span>
<span class="w">    </span><span class="o">{</span>
<span class="w">      </span><span class="s2">&quot;oid&quot;</span>:<span class="w"> </span><span class="s2">&quot;testfile2&quot;</span>,
<span class="w">      </span><span class="s2">&quot;key&quot;</span>:<span class="w"> </span><span class="s2">&quot;&quot;</span>,
<span class="w">      </span><span class="s2">&quot;snapid&quot;</span>:<span class="w"> </span>-2,
<span class="w">      </span><span class="s2">&quot;hash&quot;</span>:<span class="w"> </span><span class="m">2905889452</span>,
<span class="w">      </span><span class="s2">&quot;max&quot;</span>:<span class="w"> </span><span class="m">0</span>,
<span class="w">      </span><span class="s2">&quot;pool&quot;</span>:<span class="w"> </span><span class="m">2</span>,
<span class="w">      </span><span class="s2">&quot;namespace&quot;</span>:<span class="w"> </span><span class="s2">&quot;&quot;</span>
<span class="w">    </span><span class="o">}</span>
<span class="w">  </span><span class="o">]</span>
<span class="o">}</span>
</pre></div>
</div>
</section>
</section>
</section>
</section>



<div id="support-the-ceph-foundation" class="admonition note">
  <p class="first admonition-title">Brought to you by the Ceph Foundation</p>
  <p class="last">The Ceph Documentation is a community resource funded and hosted by the non-profit <a href="https://ceph.io/en/foundation/">Ceph Foundation</a>. If you would like to support this and our other efforts, please consider <a href="https://ceph.io/en/foundation/join/">joining now</a>.</p>
</div>


           </div>
           
          </div>
          <footer><div class="rst-footer-buttons" role="navigation" aria-label="Footer">
        <a href="../cxx/" class="btn btn-neutral float-left" title="C++17 and libstdc++ ABI" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left" aria-hidden="true"></span> Previous</a>
        <a href="../delayed-delete/" class="btn btn-neutral float-right" title="CephFS delayed deletion" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right" aria-hidden="true"></span></a>
    </div>

  <hr/>

  <div role="contentinfo">
    <p>&#169; Copyright 2016, Ceph authors and contributors. Licensed under Creative Commons Attribution Share Alike 3.0 (CC-BY-SA-3.0).</p>
  </div>

   

</footer>
        </div>
      </div>

    </section>

  </div>
  

  <script type="text/javascript">
      jQuery(function () {
          SphinxRtdTheme.Navigation.enable(true);
      });
  </script>

  
  
    
   

</body>
</html>